Online Compact Convexified Factorization Machine
نویسندگان
چکیده
Factorization Machine (FM) is a supervised learning approach with a powerful capability of feature engineering. It yields state-ofthe-art performance in various batch learning tasks where all the training data is made available prior to the training. However, in real-world applications where the data arrives sequentially in a streaming manner, the high cost of re-training with batch learning algorithms has posed formidable challenges in the online learning scenario. The initial challenge is that no prior formulations of FM could fulfill the requirements in Online Convex Optimization (OCO) – the paramount framework for online learning algorithm design. To address the aforementioned challenge, we invent a new convexification scheme leading to a Compact Convexified FM (CCFM) that seamlessly meets the requirements in OCO. However for learning Compact Convexified FM (CCFM) in the online learning setting, most existing algorithms suffer from expensive projection operations. To address this subsequent challenge, we follow the general projection-free algorithmic framework of Online Conditional Gradient and propose an Online Compact Convex Factorization Machine (OCCFM) algorithm that eschews the projection operation with efficient linear optimization steps. In support of the proposed OCCFM in terms of its theoretical foundation, we prove that the developed algorithm achieves a sub-linear regret bound. To evaluate the empirical performance of OCCFM, we conduct extensive experiments on 6 real-world datasets for online recommendation and binary classification tasks. The experimental results show that OCCFM outperforms the state-of-art online learning algorithms.
منابع مشابه
$n$-factorization Property of Bilinear Mappings
In this paper, we define a new concept of factorization for a bounded bilinear mapping $f:Xtimes Yto Z$, depended on a natural number $n$ and a cardinal number $kappa$; which is called $n$-factorization property of level $kappa$. Then we study the relation between $n$-factorization property of level $kappa$ for $X^*$ with respect to $f$ and automatically boundedness and $w^*$-$w^*$-continuity...
متن کاملA Compact Representation of Nondeterministic (Suffix) Automata for the Bit-Parallel Approach
Article history: Available online 2 February 2012 We present a novel technique, suitable for bit-parallelism, for representing both the nondeterministic automaton and the nondeterministic suffix automaton of a given string in a more compact way. Our approach is based on a particular factorization of strings which on the average allows to pack in a machine word of w bits automata state configura...
متن کاملOnline Learning for Matrix Factorization and Sparse Coding
Sparse coding—that is, modelling data vectors as sparse linear combinations of basis elements—is widely used in machine learning, neuroscience, signal processing, and statistics. This paper focuses on the large-scale matrix factorization problem that consists of learning the basis set in order to adapt it to specific data. Variations of this problem include dictionary learning in signal process...
متن کاملOnline Learning for Matrix Factorization and Sparse Coding Online Learning for Matrix Factorization and Sparse Coding
Sparse coding—that is, modelling data vectors as sparse linear combinations of basis elements—is widely used in machine learning, neuroscience, signal processing, and statistics. This paper focuses on the large-scale matrix factorization problem that consists of learning the basis set, adapting it to specific data. Variations of this problem include dictionary learning in signal processing, non...
متن کاملOnline Kernel Matrix Factorization
The problem of efficiently applying a kernel-induced feature space factorization to a largescale data sets is addressed in this thesis. Kernel matrix factorization methods have showed good performances solving machine learning and data analysis problems. However, the present growth of the amount of information available implies the problems can not be solved with conventional methods, due their...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- CoRR
دوره abs/1802.01379 شماره
صفحات -
تاریخ انتشار 2018